Moments of a Random Signal

	Consider a random process, such as the generation of a thermal noise voltage by a resistor and supposing we observe this voltage over some interval and sample it to yield a DT signal. As this is one occasion where it is necessary to distinguish carefully between a sequence and a particular value of that sequence, we will denote this signal as {x(n)}.  If we now repeat the observation, we will obtain a different {x(n)}  Each such {x(n)} is then a single realisation, or sample sequence, of the random process.  The entire set of such realisations is called the ensemble of the process.  The reader is warned that some authors reserve the notation {x(n)} for the ensemble rather than a single realisation.  To avoid our notation becoming over-cumbersome, and in the belief that no confusion results, we will use x(n) as a descriptor of the random process - as well as a particular value taken from a single realisation.

	If we can describe the ensemble by some probability law then, although a particular {x(n)} will not be predictable from its past values, its average properties will be.  In this way, we can partially yet usefully characterise the random signal.

	A very obvious statistical, average descriptor of a signal is its mean value.  The mean of the ensemble is defined as:
	
	That is, it is the mean value of a single realisation as the length of that realisation tends to infinity and so becomes identical to the ensemble.  This is sometimes called the expected value or expectation, E(), of x(n) so that:
	
	It is sometimes helpful when dealing with multiple processes to distinguish the particular random signal described by μ.  In such a case, the above value is denoted μx.

	The mean of the ensemble can be estimated from a sample sequence of length N as the sample mean:
	
	where the `hat ˆ' is used here and subsequently to indicate a statistical estimation.  For consistency with the usual notation, note that we have let the index n run from 0 to (N-1).

	The ensemble mean describes the random signal's so-called central tendency.  Another useful signal descriptor is its average fluctuation.  Since this can be negative as well as positive, we take the expected value of x2(n):

5.3.1	Cumulant Generating Functions

	Lex X be a random variable with MGF M(t).  Write:
	
	The function K(t) is called the cumulant generating function of X.  When K(t) may be expanded in a power series of the form:
	
	the coefficient of (tr/r!), Kr, is called the rth cumulant of X.  (Note that since M(0) = 1, K(0) =0, so that there is no constant term in the expansion of K(t).

	The cumulants, {Kr}, are closely related to the moments, and by expanding M(t) and comparing coefficients of powers of t we may derive expressions for each of the cumulants in terms of the moments, although such expressions rapidly become more complicated as r increases.  In particular, we have for the first cumulants:

5.3.2	Polyspectra

	The Fourier transforms of the corresponding higher order cumulants are called "polyspectra", and are defined formally as follows.

	Let {Xt} be a process stationary up to order k, and let C(s1, s2, ..., sk-1) denote the joint cumulant of order k of the set of random variables {Xt, Xt+s1, ..., Xt+sk-1, ie. C(s1, s2, ..., sk-1) is the coefficient of (z1, z2, ..., zk) in the expansion of the cumulant generating function.

	K(z1, z2, ..., zk) = log [E[exp{z1 X1 + z2Xt+s1 + ... + zkXt+sk-1}]]
	(Note that by the stationarity condition, C(s1, s2, ..., sk-1) does not depend on t).  The kth order polyspectrum, (or kth order cumulant spectrum) is defined by:

	We are assuming, of course that the above Fourier transform exists, a sufficient condition being that:

	Since the second order cumulant C(s1) is simply cov{Xt, Xt+s1}, it follows that the second order polyspectrum is exactly the same as the conventional power spectrum, ie. h2(ω)  h(ω).  Also, the third order cumulant C(s1, s2) is identical with the third order moment about the mean, ie.
	
	say (where μx = E{Xt}), and hence the third order polyspectrum may be written as:
	
	and inverting this transform we obtain:

	The function h3(ω1,ω2) is sometimes called the "bispectrum" (this terminology is due to Tukey (1959) and the fourth order polyspectrum, h4(ω1,ω2,ω3) is called the "trispectrum".  If we write:
	
	then μ(s1,s2) can be written as:

	Since the LHS is a function of s1, s2 only, so is the RHS, and it follows that E[dZx(ω1 dZx(ω2) dZx(ω3)] must vanish except along the line, ω1 + ω2 + ω3 = 0.  Hence, we may write the bispectrum in the alternative form:
	h3(ω1,ω2)dω1dω2 = E[dZx(ω1 dZx(ω2) dZx(-ω1-ω2)].

	The outstanding property of polyspectra is that all polyspectra of higher order than the second vanish when {X1} is a Gaussian process.  this follows immediately from the well known result that all joint cumulants of higher order than the second vanish for multivariate normal distribution.  Hence, the bispectrum, trispectrum, and all higher order polyspectra are identically zero if Xt is Gaussian, and these higher order spectra may thus be regarded as measures of the departure of the process from Gaussianity.

	Given two processes Xt, Yt (jointly stationary up to the appropriate order) we may similarly define "cross-polyspectra" of various orders.  For example, the third order cross-spectrum is defined by:
	
	where
	
	Writing
	
	a similar argument to that used above shows that hxxy(ω1, ω2) can be written as:

	The second order cross-polyspectrum is, of course, identical with the conventional cross-spectra density function between Yt and Xt.

	The idea of constructing the  Fourier transforms of the higher order cumulants was suggested by Kolmogorov, and polyspectra were introduced by Shiryaev (1960).  Brillinger (1965) and Brillinger and Rosenblatt (1967) have given a comprehensive treatment of the theoretical properties of polyspectra, and have discussed also the estimation of polyspectra from sample records.  (These estimation procedures are based on a generalization of the  "window" technique applied to products of the finite Fourier transform of the data.

	If a process is Gaussian then all its polyspectra (of higher order than the second) are identically zero.  Thus, if a process has a non-zero bispectrum this could be due to either:
	(i)	the process conforms to a linear model; or
	(ii)	the process conforms to a non-linear model.

	Subba Rao and Gabr (1981) have constructed two tests aimed at detecting:
	(a)	Whether the process is Gaussian - in which case, given that it is stationary with a purely continuous spectrum, it must necessarily conform to a linear model; and
	(b)	If the process is non-Gaussian, whether it conforms to a linear model.

	Case (a) is examined by testing the null-hypothesis that the bispectrum is zero at all frequencies; the test statistic is constructed from the values of the estimated bispectrum over a grid of "frequencies", and has a form similar to Hotelling's T2 statistic.  Case (b) is tested by using the result that for the linear model,
	
	where β = E(e3t) is the third moment of the {et} and gu=0, u<0.  (This result follows immediately from the independence of the {et}.  Hence, for linear models the bispectrum has the form:
	
	where
	
	But the spectral density function of the process is 
	
	and hence the function
	
	The test for linearity is based on testing the constancy of the sample values of T(ω1,ω2) (computed by replacing h3(ω1,ω2) and h(ω) by their sample estimates) over a grid of frequencies, and the test statistic again has a form similar to Hotelling's T2.

	These procedures do not, of course, provide complete tests for either Gaussianity or linearity.  It is quite possible for a non-Gaussian process to have a zero bispectrum, and a complete test would involve testing all the higher order polyspectra.  (In fact, it is clear from the above expression for h3(ω1,ω2) that this function will vanish for a linear model provided only that et has zero third moment).  Similarly, the constance of T(ω1,ω2) does not necessarily imply that the process follows a linear model.  However, it seems reasonable to suppose that in most practical situations deviations from Gaussianity or linearity would show up in the form of the bispectrum, and this is the basis of the above tests.